Skip to content

feat(runtime): Update pollrate behavior and add tests for it - #890

Open
CagriYonca wants to merge 2 commits into
mainfrom
pollrate-new-metrics
Open

feat(runtime): Update pollrate behavior and add tests for it#890
CagriYonca wants to merge 2 commits into
mainfrom
pollrate-new-metrics

Conversation

@CagriYonca

@CagriYonca CagriYonca commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Why

The tracer previously capped poll_rate to a maximum of 5 seconds,
silently resetting any higher value back to 1. This prevented customers
from benefiting from coarser poll rates (e.g. 60 s, 120 s) to reduce
observability cost.

Additionally, GC metrics were collected using gc.get_count() and
gc.get_threshold(), which do not reflect actual GC activity. The correct
source is gc.get_stats(), which tracks cumulative collections,
collected, and uncollectable counters per generation — consistent with
the OpenTelemetry CPython semantic conventions
(cpython.gc.collections, cpython.gc.collected_objects,
cpython.gc.uncollectable_objects) and the
OpenTelemetry Python contrib system_metrics instrumentation,
both of which source these metrics from gc.get_stats().

Finally, the host agent timeout detection used a fixed 60-second window,
causing spurious disconnects when poll_rate exceeded 30 seconds.

What

  • options.py: Replaced MAX_POLL_RATE = 5 with a VALID_POLL_RATES
    list [1, 5, 10, 20, 30, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600].
    set_poll_rate() now rounds to the nearest valid value instead of
    rejecting anything outside {1, 5}.

  • runtime.py: Switched GC collection from gc.get_count() /
    gc.get_threshold() to gc.get_stats(). Reports per-generation deltas
    for collections, collected, and uncollectable (keys
    gc.collections0-2, gc.collected0-2, gc.uncollectable0-2).

  • host.py: Timeout window is now max(60, poll_rate * 2) seconds to
    avoid false resets at high poll rates. Added _send_heartbeat() — a
    lightweight HEAD request sent when no metrics/spans/profiles are
    produced in a cycle, keeping last_seen current between polls.

Testing

Unit tests are included for all changed behaviours in:
tests/test_options.py, tests/collector/helpers/test_collector_runtime.py,
tests/collector/test_host_collector.py, tests/agent/test_host.py

@CagriYonca CagriYonca self-assigned this Aug 7, 2026
@CagriYonca
CagriYonca requested a review from a team as a code owner August 7, 2026 10:40
@CagriYonca
CagriYonca marked this pull request as draft August 7, 2026 12:48
@CagriYonca
CagriYonca force-pushed the pollrate-new-metrics branch 2 times, most recently from 457617c to b7ce752 Compare August 11, 2026 15:09
Signed-off-by: Cagri Yonca <cagri@ibm.com>
@CagriYonca
CagriYonca force-pushed the pollrate-new-metrics branch from b7ce752 to 07c716c Compare August 11, 2026 15:15
@CagriYonca CagriYonca changed the title feat(runtime): Add pauseMs and runCount metrics to runtime feat(runtime): Update pollrate behavior and add tests for it Aug 11, 2026
@CagriYonca
CagriYonca marked this pull request as ready for review August 11, 2026 17:05
…esponding metrics.

Signed-off-by: Cagri Yonca <cagri@ibm.com>
@CagriYonca
CagriYonca force-pushed the pollrate-new-metrics branch from 07c716c to df71434 Compare August 13, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant